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CLAIMS 

What is claimed is: 

1 1 . A method of controlling the operation of a speech recognition device, 

2 comprising the steps of: 

3 recording at least one frame of a video image of speech articulators of a user while the 

4 user is speaking; 

5 recording acoustic properties of speech that occurs concurrent with the recording of the at 

U 6 least one video frame; 

y 

yi 7 identifying acoustic properties of speech that would be expected to be generated by a 

f!i 

M 8 condition of the speech articulators recorded in the at least one frame of the video image; and 

s.i 

Q ? 9 comparing the identified acoustic properties of speech with the recorded acoustic 

fiO properties to determine whether the speech of the recorded properties emanated from the user. 

p 1 2. The method of claim 1 further comprising the step of: 

R 5 

2 activating the speech recognition device when there is a match between the acoustic 

3 properties of speech which would be expected to be generated by the condition of the speech 

4 articulators recorded in the at least one frame of video image with the acoustic properties of 

5 speech recorded concurrent with the recording of the at least one video frame. 

1 3. The method of claim 2 further comprising the step of: 

2 maintaining the speech recognition device active for a preset time interval after being 

3 activated. 

1 4. The method of claim 3 further comprising the step of: 

13 



By Express Mail # EV01 1853923US 

2 maintaining the speech recognition device activate beyond the end of the preset time 

3 interval upon obtaining a match between the acoustic properties of speech which would be 

4 expected to be generated by the condition of the speech articulators recorded in a subsequently 

5 recorded frame of a video image with the acoustic properties of speech recorded concurrent with 

6 the recording of the subsequently recorded video frame before the fixed period of time expires. 

1 5. The method of claim 1 wherein a camera is used to record the video image of the 

2 speech articulators of the user. 



\q 1 6. The method of claim 1 wherein a microphone is used to record the acoustic properties 

flj 

gn 2 of s peech of the user. 
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O 
1 



7. The method of claim 1 wherein a handheld device contains a microphone for recording 



JJj 2 the acoustic properties of speech of the user and a camera for recording the video image of 



% 3 speech articulators of the user. 



2 8. A method of controlling the operation of a speech recognition device comprising the 

3 steps of: 

4 recording a series of frames of video images of speech articulators of a user while 

5 speaking; 

6 recording acoustic properties of speech that occurs concurrent with the recording of each 

7 of the series of video frames; 
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identifying each frame of the series of frames of video images with the acoustic 
properties of sounds which are obtained concurrent with the recording of the series of video 
frames; 

examining the video frames for a face; 

examining the video frames that have a face for a change of the speech articulators of the 

face; 

identifying acoustic properties of speech that would be expected to be generated by a 
condition of the speech articulator recorded in the video frame that has a changed speech 
articulator; 

identifying the recorded acoustic properties of speech that occurred at the time that the 
video frame of a face having a change of speech articulators was obtained; and 

comparing the identified acoustic properties of speech that occurred at the time that the 
video frame of a face having a change of speech articulators with the identified acoustic 
properties that would be expected to be generated to determine whether the speech of the 
identified acoustic properties emanated from the user. 

9. The method of claim 8 further comprising the step of: 

activating the speech recognition device when there is a match between the identified 
acoustic properties of speech that occurred at the time that the video frame of a face having a 
change of speech articulators with the identified acoustic properties that would be expected to be 
generated concurrently wit the video frame. 

10. The method of claim 9 further comprising the step of: 
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maintaining the speech recognition device activated for a preset time interval after 
activating the speech recognition device. 

11. The method of claim 10 further comprising the step of: 

deactivating the speech recognition device at the end of the preset time interval in the 
absence of the occurrence of a subsequent match between the identified acoustic properties of 
speech that occurred at the time that the video frame of a face having a change of speech 
articulators with the identified acoustic properties that would be expected to be generated 
concurrently with the video frame.. 

12. Apparatus for controlling the operation of a speech recognition device comprising; 
video means for recording at least one video image of the speech articulators of a user 

and analyzing the video image to identify the acoustic properties of speech that would be 
expected to be generated by the condition of the speech articulators; 

acoustic means for recording acoustic properties of speech by the user that occur 
concurrently with the recording of the at least one video image; 

comparing means for comparing the acoustic properties of speech that would be expected 
to be generated by the condition of the speech articulators with the recorded acoustic properties 
of speech by the user, and 

control means to activate the speech recognition device when the comparing means identifies a 
match. 

13. Apparatus according to claim 12 further comprising: 
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a video signal processing means for analyzing the at least one video image to identify the 
acoustic properties of speech that would be generated by the condition of the speech articulators. 

14. The apparatus of claim 12 wherein the video means is a video camera and the 
acoustic means is a microphone 

15. The apparatus of claim 14 wherein the video camera and microphone are in a 
handheld device. 

16. An article comprising: 

a computer program in a machine readable medium wherein the computer program 
executes on a suitable platform to control the operation of a speech recognition unit and is 
operative to automatically analyze at least one video image to detect a change of the speech 
articulators of the face of a user and generate a characteristic of speech which can be made by the 
shape of the speech articulators. 

17. The article of claim 16 wherein the computer program automatically compares the 
generated speech with actual speech made at the time that the video image was obtained to 
determine if the actual speech is the speech of the user at the time that the video image was 
made. 
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